Strong polynomiality of the Gass-Saaty shadow-vertex pivoting rule for controlled random walks

We consider the subclass of linear programs that formulate Markov Decision Processes (mdps). We show that the Simplex algorithm with the GassSaaty shadow-vertex pivoting rule is strongly polynomial for a subclass of mdps, called controlled random walks (CRWs); the running time is O(|S| · |U |), where |S| denotes the number of states and |U | denotes the number of actions per state. This result improves the running time of Zadorojniy et al. [30] algorithm by a factor of |S|. In particular, the number of iterations needed by the Simplex algorithm for CRWs is linear in the number of states and does not depend on the discount factor.